Recently, in addition to an optical fiber and ADSL in a fixed network, also in a mobile network, spot wireless technologies such as WLAN and an Ad-Hoc network have been widespread. Then, a ubiquitous network environment in which network connection can be made everywhere is becoming realized. Furthermore, also in the field of communication terminals, devices having various abilities, for example, Internet home appliances and public-local dual terminals have been developed. There has been a strong demand for a seamless service capable of providing continuous services by freely switching various access networks and devices having various abilities in a ubiquitous network environment.
As a technology for realizing a seamless service, three technologies have been proposed. The first technology is referred to as network seamless. In this technology, one mobile terminal can switch between different networks seamlessly in accordance with the movement, and the like, so that the communication can be continued. The mobile terminal has a plurality of network interfaces and can switch access networks freely in accordance with the movement. The network seamless is also referred to as terminal mobility.
The second technology is referred to as device seamless. In this technology, a user can seamlessly switch terminals in use. Thus, a session used in one terminal can be continuously used in another terminal. This technology is also referred to as session mobility because the session is moved to another terminal.
The third technology is referred to as contents seamless. In this technology, a media format, a media type, and the like, can be switched seamlessly. Thus, display and reproduction can be carried out by switching media in accordance with a network band, device ability, a peripheral environment, and the like. The contents seamless may be also referred to as media adaptive (media adaptation).
These three technologies are very important in realizing seamless service, and they can exhibit independent effects, respectively. Therefore, these technologies can be used alone or in a combination thereof.
A seamless service employing device seamless technology (hereinafter, referred to as “device switching”) is thought to include the following situation. For example, as soon as a user, who comes home while viewing a streaming movie by using a mobile terminal such as a portable telephone, enters the living room of his/her house, he/she instantly switches the display to a large size television display and views the continuation of the movie.
In general, a display of a mobile terminal is small and not preferable for viewing movies. Thus, when a suitable device capable of continuously providing service is discovered in the surrounding in accordance with the movement and if a device can be switched to the device, very high applicability is achieved.
A technology for satisfying such a demand is described in Japanese Patent Unexamined Publication No. 2003-304251. FIG. 26 shows a conventional device switching method described in this patent publication.
In FIG. 26, firstly, mobile terminal (PDA) 121 possessed by a user detects neighboring terminal (PC) 122 by using short distance radio and automatically obtains an address on the communication network of neighboring terminal (PC) 122. Then, mobile terminal (PDA) 121 becomes a transfer source terminal and transfers a communication session of the application level during communication with communication destination terminal (PC) 300 to neighboring terminal (PC) 122 that is a transfer destination terminal.
Next, a conventional device switching system is described with reference to drawings.
FIG. 19 is a diagram showing a configuration of a conventional device switching system. This system includes mobile node (hereinafter, referred to as “MN”) 1301 that is a switching source device; counterpart node (hereinafter, referred to as “CN”) 1302 that is a communication counterpart; the Internet 1303; device group 1304 that is a switching destination device for device switching, for example, plasma TV, stereo, and PC; and personal area network (hereinafter, referred to as “PAN”) 1305 for mutually connecting between MN 1301 and device group 1304.
MN 1301 and device group 1304 can freely communicate with each other in PAN 1305 by using PAN 1305. Furthermore, CN 1302 and MN 1301 communicate with each other via the Internet 1303. CN 1302 is, for example, a streaming server. MN 1301 streaming receives a movie distributed from CN 1302. Note here that a network between MN 1301 and CN 1302 is not necessarily the Internet. It may be, for example, a third generation mobile communication network (3GPP network).
Next, a configuration inside the device of MN 1301 that is a switching source device for realizing device switching is described with reference to FIG. 20.
MN 1301 includes application 1401, at least one network I/F 1402, output section 1403 for outputting a display, input section 1404 as an interface (I/F) with respect to a user, operating system 1405, media sending section 1406 communicating with an external device (not shown) via operating system 1405, media receiving section 1407, media buffer 1408 for storing media data received by media receiving section 1407, decoder 1409 for decoding data when media data are compressed, signaling section 1410 for controlling a session, service discovery section 1411 for discovering a service provided by an external switching destination device (not shown), and middleware 1412 for controlling so that media sending section 1406 transfers received media data to the switching destination device.
As mentioned above, it can be thought that MN 1301 is one of the devices of device group 1304 and a switching destination device has the same configuration as that of MN 1301. However, when it is not necessary that the switching destination device operate as a switching source device, application 1401, input section 1404 and media sending section 1406 may be omitted from the configuration.
FIG. 21 is a view showing a message sequence between MN 1301 and switching destination device group 1304. FIG. 22 is a flowchart showing the similar message sequence. Herein, device group 1304 includes N pieces of devices from device 1 to device N. Furthermore, MN 1301 holds another session with CN 1302 although not shown in the drawing. In FIG. 21, a user denotes a user who currently uses MN 1301, and the user can give an instruction to MN 1301 via input section 1404. Furthermore, an arrow in FIG. 21 indicates that a message is sent in the direction from the starting point to the ending point.
An operation of a conventional device switching system is described with reference to the sequence diagram shown in FIG. 21 and the flowchart shown in FIG. 22.
A user who intends to switch devices makes an instruction of request to MN 1301 to display names of switching destination candidate devices (U1) by using input section 1404 of MN 1301 (step S1601).
Next, middleware 1412 that receives the instruction through input section 1404 of MN 1301 sends service discovery request (M1) to devices 1 to N in PAN 1305 at one time through service discovery section 1411 (step S1602).
Next, when service discovery section 1411 of the switching destination device receives the service discovery request from MN 1301, it replies service discovery response (M2) to MN 1301 (step S1603). As a service discovery protocol used by service discovery section 1411, existing protocols such as SSDP (Simple Service Discovery Protocol) of UPnP (Universal Plug and Play), SDP and SLP (Service Location Protocol) of Bluetooth (trademark) are used.
Service discovery section 1411 of MN 1301 receives service discovery response (M2) from the switching destination device and notifies middleware 1412 of it. Middleware 1412 generates a switching destination candidate F device list including devices capable of providing service based on the notification, and the list is presented (U2) to a user by output section 1403 via operating system 1405 (step S1604). Note here that the switching destination candidate device list is a list including identifiers (for example, a device name) specifying devices.
The user selects switching destination device (U3) from the switching destination candidate device list by input section 1404 of MN 1301 (step S1605). The selected switching destination device (hereinafter, referred to as “switching destination determined device”) is notified to middleware 1412. The sequence diagram of FIG. 21 illustrates a case where device 2 is selected as a switching destination device is described.
Middleware 1412 of MN 1301 establishes a session with selected device 2 in PAN 1305, and in order to transmit media, it sends session establishment—media processing preparation request (M3) to device 2 through signaling section 1410 (step S1606). At the same time, middleware 1412 of MN 1301 notifies media receiving section 1407 and media sending section 1406 to start preparation for transferring the received media data to a switching destination device. As signaling section 1410, an existing protocol such as SIP (session initiation protocol) can be used.
When signaling section 1410 of switching destination device 2 receives session establishment—media processing preparation request (M3) from MN 1301, middleware 1412 of device 2 instructs media receiving section 1407, media buffer 1408 and decoder 1409 to start media processing preparation. When the media processing preparation is completed, middleware 1412 of device 2 sends session establishment—media processing preparation response (M4) to MN 1301 through signaling section 1410 (step S1607).
When signaling section 1410 of MN 1301 receives session establishment—media processing preparation response (M4) from device 2, middleware 1412 of MN 1301 notifies media sending section 1406 that media receiving section 1407 starts transmission (D1) of the media data received from CN 1302 to device 2 (step S1608).
Media receiving section 1407 of device 2 receives media data transferred by MN 1301 and starts storing the media data into media buffer 1408. When predetermined media data are stored in media buffer 1408, middleware 1412 notifies decoder 1409 to start a decoding operation when the media data are compressed. The decoded data are output (D2) to output section 1403 via operating system 1405 (step S1609). On the other hand, when media data are not compressed, the media data stored in media buffer 1408 are output to output section 1403 via operating system 1405.
As mentioned above, it is possible to realize device switching from MN 1301 to switching destination device 2. As a result, the media data that have been output to output section 1403 of MN 1301 are output from output section 1403 of device 2. In this series of switching sequence, a time that elapses from the time when a user intends to switch devices and instructs MN 1301 to display switching destination candidate device request (U1) to the time when data are output (D2) to output section 1403 of device 2 is a time necessary for switching. As the time necessary for switching is shorter, the user's waiting time in switching is reduced and more seamless service can be provided.
However, in the device switching in accordance with such a conventional technology, several seconds are required from service discovery request (M1) to the reception of service discovery response (M2). Furthermore, several seconds are required from session establishment—media processing preparation request (M3) to the reception of session establishment—media processing preparation response (M4). As a result, a time necessary for switching, that is, a user's waiting time becomes considerably long. Thus, for example, in the case where a user is viewing a soccer game in real time, the user may miss watching an important moment such as a goal scene because of the device switching.
Then, in order to reduce the time necessary for switching devices, a document “Mobile Multimedia Middleware for Seamless Service” (Ken Ohta et al., study report of Information Processing Society of Japan: “Mobile Computing and Wireless Communication” No. 18-35, Sep. 7, 2001, p 261-268) discloses two technologies, that is, a technology in which service discovery has been periodically carried out in advance (technology A) and a technology in which session establishment and media processing preparation have been carried out previously with respect to all devices obtained as a result of the service discovery (technology B).
With such technologies, it is possible to reduce the time necessary for switching devices, which is described with reference to drawings. The internal configuration of MN 1301 or the terminal of a switching destination device in the document “Mobile Multimedia Middleware for Seamless Service” is exactly the same as that described in FIG. 20 except for an operation of middleware 1412. Then, in order to distinguish this middleware from middleware 1412 shown in FIG. 20, the middleware of the document “Mobile Multimedia Middleware for Seamless Service” is referred to as a “high speed middleware.”
FIG. 23 shows a message sequence between MN 1301 and switching destination device group 1304 in the document “Mobile Multimedia Middleware for Seamless Service.” FIG. 24 is a flowchart showing an operation of a switching source device in the document. FIG. 25 shows a flowchart of an operation of a switching destination device. Herein, device group 1304 includes N pieces of switching destination devices from device 1 to device N. Furthermore, although not shown, MN 1301 holds another session with CN 1302.
Next, operations of the switching source device and the switching destination device are described. Hereinafter, in these operations, the same reference numerals are given to the same component elements as those shown in FIG. 20.
Firstly, an operation of the switching source device is described with reference to the flowchart shown in FIG. 24.
When a session with CN 1302 is continued (“YES” in step S1801), in order to carry out service discovery with respect to surrounding device group 1304, the high speed middleware periodically sends service discovery request (M1) to devices 1 to N in PAN 1305 at one time through service discovery section 1411 (step S1802). On the other hand, when the session with CN 1302 is not continued (“NO” in step S1801), the following operation is terminated.
Service discovery section 1411 receives service discovery response (M2) from the switching destination device and notifies the high speed middleware of it (step S1802). Based on the notification, the high speed middleware generates a switching destination candidate device list of devices capable of providing a service (step S1803) and holds it therein. When a switching destination candidate device list already exists, the high speed middleware updates the list.
The high speed middleware sends session establishment—media processing preparation request (M3) through signaling section 1410 to all the devices that are switching destination candidates described in the switching destination candidate device list in order to establish a session and carry out media transmission (step S1804). Herein, the number of the devices that become the switching destination candidates is n (1≦n≦N).
At the same time, the high speed middleware notifies media receiving section 1407 and media sending section 1406 to start preparation for transferring the received media data to a switching destination candidate device (step S1805).
When signaling section 1410 receives session establishment—media processing preparation response (M4) from the device in the switching destination candidates (step S1806), the high speed middleware is in a waiting state for a predetermined time.
While the high speed middleware is in a waiting state, when an instruction of switching destination candidate device display request (U1) is not sent from a user (“NO” in step S1807), the service discovery is carried out again. On the other hand, when an instruction of switching destination candidate device display request (U1) is sent from a user (“YES” in step S1807), the high speed middleware presents the switching destination candidate device list held therein to a user (U2) through output section 1403 (step S1808).
The user selects a switching destination device (U3) from the switching destination candidate device list through input section 1404 (step S1809). Input section 1404 notifies the high speed middleware of the name of the selected switching destination determined device via operating system 1405. This flowchart illustrates the case in which device 2 is selected.
The high speed middleware notifies media sending section 1406 to start transmitting (D1) media data received by media receiving section 1407 from CN 1302 (step S1810). Media sending section 1406 starts transferring the designated media data to switching destination determined device 2.
The high speed middleware sends session disconnection request (M5) through signaling section 1410 to devices other than device 2 among the n pieces of switching destination candidate devices that are switching destination candidates (step S1811). The high speed middleware receives session disconnection response (M6) from these devices and completes the switching operation.
Next, operations of the switching destination candidate device and the switching destination device as a switching destination determined device are described with reference to the flowchart shown in FIG. 25.
Service discovery section 1411 of the switching destination device, which receives service discovery request (M1), sends service discovery response (M2) to MN 1301 (step S1901).
Signaling section 1410 receives session establishment—media processing preparation request (M3) from MN 1301 (step S1902). Next, the high speed middleware makes an instruction to media receiving section 1407, media buffer 1408 and decoder 1409 to start preparation for media processing (step S1903).
When the preparation for media processing is completed, the high speed middleware sends session establishment—media processing preparation response (M4) to the switching source device through signaling section 1410 (step S1904).
Media receiving section 1407 of switching destination device 2 receives media data transferred from the switching source device (step S1905) and starts storing the media data into media buffer 1408 (step S1906). When predetermined data are stored in media buffer 1408, when the media data are not compressed (“YES” in step S1907), the high speed middleware sends the data stored in media buffer 1408 to output section 1403 via operation system 1405 (step S1909). On the other hand, when the media data are compressed (“NO” in step S1907), the high speed middleware notifies decoder 1409 to start a decoding operation (step S1908) and sends the decoded data to output section 1403 via operating system 1405. Output section 1403 starts outputting the received media data (D2) (step S1909) and completes the switching operation.
On the other hand, when signaling section 1410 of a device that does not become a switching destination determined device receives session disconnection request (M5) from the switching source device, the high speed middleware makes an instruction to media receiving section 1407, media buffer 1408 and decoder 1409 to terminate the preparation for media processing (step S1910).
When media receiving section 1407, media buffer 1408 and decoder 1409 terminate the preparation for media processing, the high speed middleware sends session disconnection response (M6) to the switching source device through signaling section 1410 (step S1911) and terminates the switching operation.
As mentioned above, according to the technology described in the document “Mobile Multimedia Middleware for Seamless Service,” it was possible to switch from a switching source device to a switching destination device at a high speed.
That is to say, before an instruction of a switching destination candidate device display request is sent from a user (step S1807), the operations from service discovery request (M1) to the reception of service discovery response (M2) (step S1802 to step S1803 and step S1901) have been periodically carried out (above-mentioned technology A) with respect to the display request of a switching destination candidate device from a user, a substantial operation time at the time of switching can be made to be zero. Thus, the time necessary for presenting a device list can be reduced.
Furthermore, before an instruction to select a switching destination device is made from a user (step S1809), the operations from session establishment—media processing preparation request (M3) to the reception of session establishment—media processing preparation response (M4) (step S1804 to step S1806 and step S1902 to step S1904) have been carried out with respect to all the switching destination candidate devices that are switching destination candidates obtained by the service discovery (above-mentioned technology B), a substantial operation time at the time of device switching can be made to be zero. Thus, the time necessary to start an output of media data at the selected switching destination device can be reduced.
As a result, a time necessary for switching, that is, a time that elapses from the time when a user intends to switch devices and makes an instruction of the switching destination candidate device display request (U1) to the switching source device to the time when media data are output (D2) to the output section of the selected switching destination device is significantly reduced.
However, in the technology described in the document “Mobile Multimedia Middleware for Seamless Service,” from the time the selected switching destination device starts to receive media data to the time when the output section of the switching destination device starts outputting media data (step S1905 to step S1909), about several hundreds milliseconds are required, and therefore a sufficient seamless service cannot be still realized. Furthermore, in order to avoid the delay, the sending section of the switching source device sends data from a media data portion newly received by the receiving portion (step S1810), media data that have been stored in the media buffer of the switching source device and that have not yet output at switching source device are not transferred to the switching destination device. Therefore, since the output portion of the switching destination device cannot output the media data portion, the output media data have defectiveness, resulting in generating discontinuous output.